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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

• Extensions of time may be available under the provisions of 37 CFR 1.136(a). tn no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- tf NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)13 Responsive to communication(s) filed on 2 April 2003 . 
2a)(3 This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-26 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) fE Claim(s) 1-26 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) S The drawing(s) filed on 26 April 2001 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) [3 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)E9AII b)D Some*c)D None of: 

1 .(3 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). 



2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-152) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 

1. Claims 1-26 have been presented for examination based on applicants 
arguments filed on 3 April 2003. Claims 1-26 remain rejected by the examiner. 

Response to Arguments 

2. Applicants arguments filed on 3 April 2003 have been fully considered but they 
are not persuasive. 

Regarding applicants response to 112(1) rejections : Applicants have argued that 
independent claims 1, 13, and 25 gain enablement for the claimed evaluation and 
control unit for comparing component information data with real picture data via 

the text contained on pages 15, line 19 through 18, line 20 of the specification. The 
examiner respectfully disagrees. For example, page 16, line 5 recites; "The evaluation- 
and-control-unit 5 converts the information contained in the source 201 of the image 
memory 20 into a prepared source 51. However, the specification never sufficiently 
discloses how this is accomplished. The text beginning on page 16, line 7 attempts to 
describe the process but does not disclose the following : 

How is the geometric assignment 54 used to describe which geometric elements of 

a component 61 could be matched with the geometric elements for a source 

assignment 52? 

How is the geometry assigned? 

What geometry is assigned? 

What are the geometric elements?? 
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On page 18 reference is made to either the pixels 202 or the CAD elements 205 
being converted into geometric elements (e.g., point 55, line 56, curve 57) which are 
used by the evaluation-and-control-unit in assigning the selected components but never 
discloses how the conversion or the assignment is performed. 

Accordingly, the examiner assets that one skilled in the art would be at odds to 
determine how to make and/or use the claimed invention without undue 
experimentation . 

The examiner maintains the 112(1) rejections. 

Regarding applicant's response to 103(a) rejections : Applicants are reminded 
that In the previous office action the examiner made the following statement on page 4 
of paper #7. 

"While the specification for the claimed invention is delinguent in the areas cited 
above (see 1 12(1) rejections), the examiner has made prior art rejections based on the 
limited scope of the information contained in the specification. " 

In response to applicant's arguments against the references individually, one 
cannot show nonobviousness by attacking references individually where the rejections 
are based on combinations of references. See In re Keller, 642 F.2d 413, 208 
USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 
1986). It is noted that appellant's arguments regarding prior art have essentially only 
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recited the prior art teaching followed by a recitation of the claims without pointing out 
the patentable distinction between the claimed invention and the prior art . 

In response to applicants argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies 
(i.e., an image recording system (page 5, line 8), functional interplay between 
objects (page 9, Iine16) compared with the behavior of a virtual model (page 9, line 
17)) are not recited in the rejected claim(s). Although the claims are interpreted in light 
of the specification, limitations from the specification are not read into the claims . 
See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Applicants 
have also argued "exemplary" embodiments of prior art. Such arguments are not 
considered. 

In response to applicant's arguments, the recitation "generating the virtual 
image of a real installation" has not been given patentable weight because the 
recitation occurs in the preamble. A preamble is generally not accorded any patentable 
weight where it merely recites the purpose of a process or the intended use of a 
structure, and where the body of the claim does not depend on the preamble for 
completeness but, instead, the process steps or structural limitations are able to stand 
alone. See In re Hirao, 535 F.2d 67, 190 USPQ 15 (CCPA 1976) and Kropa v. Robie, 
187F.2d 150, 152, 88 USPQ 478, 481 (CCPA 1951). 

In response to applicant's argument that there is no suggestion to combine the 
references, the examiner recognizes that obviousness can only be established by 
combining or modifying the teachings of the prior art to produce the claimed invention 
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where there is some teaching, suggestion, or motivation to do so found either in the 
references themselves or in the knowledge generally available to one of ordinary skill in 
the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988) and In re 
Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, prior art (Marks) 
provides sufficient motivation as cited under 103(a) rejections, (see Marks CL1-L42, 
L60) 

Claim Rejections - 35 USC §112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

3. Claims 1-12 are rejected under 35 U.S.C. 112, first paragraph, as containing 
subject matter which was not described in the specification in such a way as to 
enable one skilled in the art to which it pertains, or with which it is most nearly 
connected, to make and/or use the invention. 

Specifically, in independent claim 1 applicants are claiming an evaluation and 
control unit for comparing component information data with real picture data but the 
specification does not disclose an algorithm or technique for comparing component 
information data with real picture data. Neither the operation of the evaluation and 
control unit, nor the process of identifying components in picture data, nor the 
claimed deriving hypotheses, is sufficiently described in the specification to allow one 
skilled in the art to make and/or use the invention. 
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While page 4, line 5 of the specification state that the evaluation and control unit 
controls the process of generating the virtual installation model and controls an 
"automatic function" (page 5, line 13), it does not sufficiently describe functional 
operation either the process or the "automatic function". Obviously, one skilled in the art 
would not be able to create an "automatic function" without specific details on how the 
automatic function operates. Merely stating that the evaluation and control unit 
"processes component data" (page 8, line 8) and "performs image analysis" (page 8, 
line 20) does not provide a sufficient description to allow one skilled in the art to make 
and/or use the invention. Dependent claims inherit these defects. 

Claims 13-26 are rejected under 35 U.S.C. 112, first paragraph, because the 
specification, while being enabling for "generating picture data" and "storage to 
store", does not reasonably provide enablement for "comparing picture data to 
identify installation components" (claim 13) or "a processing unit to compare 
components". The specification does not enable any person skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use 
the invention commensurate in scope with these claims. 

Specifically, independent claim 13 claims comparing picture data to identify 
installation components within the picture data but again the specification does not 
disclose an algorithm or technique for performing the comparison or identifying the 
components. 
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Independent claim 25 claims a processing unit to compare components but 

does not disclose the process for comparing components sufficient to allow one skilled 
in the art to make and/or use the invention. 

Dependent claims inherit these defects. 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 

USPQ 459 (1966), that are applied for establishing a background for determining 

obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

4. Claims 1-26 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

U.S. Patent 6,025,847 issued to Marks in view of U.S. Patent 6,477,266 issued to 

Asar in further view of U.S. Patent 5,552,984 issued to Crandall et al. 

While the specification for the claimed invention is delinquent in the areas cited 
above (see 112(1) rejections), the examiner has made prior art rejections based on the 
limited scope of the information contained in the specification. 
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Independent claim 1 is drawn to: 

Generating an image of installation model by: 
Memory (1 st ) for storing picture data 
Memory (2 nd ) for storing component information 
Memory (3 rd ) for storing virtual installation model 

Evaluation and control unit for comparing component information data with real picture data 
Identifying components in picture data as installation components 
Deriving a hypotheses for identified components in picture data 
Generating respective installation components in virtual installation model 

Regarding independent claim 1 : Marks teaches a system and method of 
generating a 3D (virtual) model from an image (picture) using a computer system. 
The image is of an arrangement of physical objects (components) where primitives 
(geometric) representing a portion of a physical object (a component) in the image are 
specified as a set of parameters that correspond locations in the image model. Marks 
further discloses incorporating a memory space for storing picture data, component 
(object) information. (Abstract, Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, 
CL7-L35-67, CL7-L5-50) 

Marks does not explicitly teach comparing component information data with real 
picture data. 

Asar teaches a system and method for identifying components by comparing 
component information data with real picture data and incorporating multiple 
memories for storing picture data and component information. Asar further discloses 
evaluating (deriving a hypotheses via the image control system) identified components 
in the picture data by determining defects and placement errors and generating a 
respective image of the installation of components. (Abstract, Summary of Invention, 
Figs. 9-25, CL3-L29-43, CL5-L12-25, CL6-L10-35) 
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Marks mentions, but further does not explicitly teach virtual components 
(installation). 

Crandall teaches generating a virtual model of a real system using virtual 
components (installation components) from a library of components representing the 
total virtual system (installation). (Abstract Summary of Invention, Figs. 1a&b, 9, CL2- 
L5-Y7, CL4-L1-29) 

It would have obvious to one of ordinary skill in the art at the time the claimed 
invention was made to modify the teachings of Marks relating to a system and method 
for generating a 3D (virtual) model from an image (picture), with the teachings ofAsar 
relating to identifying components by comparing component information data with 
real picture data, and to further modify the teachings of Marks with the teachings of 
Crandall relating to generating a virtual model of a real system using virtual 
components (installation components) from a library of components representing the 
total virtual system (installation), to realize a device and method for generating a virtual 
model of an installation. An obvious motivation exists since, as referenced by prior art, 
creating a 3D (virtual) model of physical objects provides more efficient detection of 
discrepancies between the model and the actual image. 

Regarding dependent claims 2-4 : Asar performs image analysis on picture data 
as previously cited above. (CL2-L5-11) Crandall teaches using virtual components 
(installation components) from a library of components representing the total virtual 
system (installation) (CL2-L5-11 ). Marks teaches objects being located by geometric 
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information and multiple window views of picture data and 3D (virtual) model views. 
(CL4-L1 0-25, Figs. 2-7) 

Regarding dependent claims 5-9 : Marks discloses building a 3D (virtual) model 
using the well-known techniques of "dropping", "clicking and dragging", and "rubber 
banding" in the manipulation of primitives based on geometric information (CL4-L42). 
Marks also teaches the evaluation of structural components (by function) to assign 
primitives (add components) in a 3d (virtual) installation model. (Figs. 9-25, CL3-L29-43, 
CL5-L12-25, CL6-L10-35) 

Regarding dependent claims 10-12 : Marks teaches the "automatic" component 

location (CL7-42) and a system incorporating a digital camera, digitized photographs 

(picture data), and a CAD system with memory and a multiple perspective view display. 

(Abstract, Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, CL7-L35-67, CL7-L5- 

50) 

Independent claim 13 is drawn to: 

Generating an image of real installation model by: 
Generating picture data 

comparing component information data with picture data 
Identifying components in picture data as installation components 

Regarding independent claim 13 : As previously cited, Marks teaches a system 
and method of generating a 3D (virtual) model from an image (picture) using a 
computer system. The image is of an arrangement of physical objects (components) 
where primitives (geometric) representing a portion of a physical object (a component) 
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in the image are specified as a set of parameters that correspond locations in the 
image model. (Abstract Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, CL7~ 
L35-67, CL7-L5-50) 

Marks does not explicitly teach comparing component information data with real 
picture data. 

Asar teaches a system and method for identifying components by comparing 
component information data with real picture data and incorporating multiple 
memories for storing picture data and component information. Asar further discloses 
evaluating (deriving a hypotheses via the image control system) identified components 
in the picture data by determining defects and placement errors and generating a 
respective image of the installation of components. (Abstract, Summary of Invention, 
Figs. 9-25, CL3-L29-43, CL5-L12-25, CL6-L10-35) 

Marks mentions, but further does not explicitly teach virtual components 
(installation). 

Crandall teaches generating a virtual model of a real system using virtual 
components (installation components) from a library of components representing the 
total virtual system (installation). (Abstract, Summary of Invention, Figs. 1a&b, 9, CL2- 
L5-11, CL4-L1-29) 

It would have been obvious to one of ordinary skill in the art at the time the 
claimed invention was made to modify the teachings of Marks relating to a system and 
method for generating a 3D (virtual) model from an image (picture), with the 
teachings of Asar relating to identifying components by comparing component 
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information data with real picture data, and to further modify the teachings of Marks 
with the teachings of Crandall relating to generating a virtual model of a real system 
using virtual components (installation components) from a library of components 
representing the total virtual system (installation), to realize a device and method for 
generating a virtual model of an installation. An obvious motivation exists since, as 
referenced by prior art, creating a 3D (virtual) model of physical objects provides more 
efficient detection of discrepancies between the model and the actual image. 

Regarding dependent claims 14-16 : As also previously cited, Asar teaches image 
analysis on picture data as previously cited above. (CL2-L5-11) Crandall teaches using 
virtual components (installation components) from a library of components 
representing the total virtual system (installation) (CL2-L5-11) Marks teaches the 
manipulation of primitives based on geometric information (CL4-L42) and multiple 
window views of picture data and 3D (virtual) model views. (CL4-L10-25, Figs. 2-7). 

Regarding dependent claims 1 7-24 : Marks discloses building a 3D (virtual) model 
using the well-known techniques of "dropping", "clicking and dragging", and "rubber 
banding" in the manipulation (matching) of primitives based on geometric information 
(CL4-L42). Marks also teaches the evaluation of structural components (by function) 
to assign primitives (add components) in a 3d (virtual) installation model. (Figs. 9-25, 
CL3-L29-43, CL5-L12-25, CL6-L10-35) It further would have been obvious, and 
necessary, to "select" and "drag" installation components since all modern GUI based 
CAD system operate in this manner. Marks further teaches "automatic" component 
location (CL7-42) and a system incorporating a digital camera, digitized photographs 
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(picture data), and a CAD system with memory and a multiple perspective view display. 
(Abstract Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, CL7-L35-67, CL7-L5- 
50) 

Regarding claims 25 and 26 : Claims 25 and 26 merely claim the virtual model of 
a facility that includes the same limitations as disclosed in claims 1-24. Claims 25 and 
26 are therefore rejected using the same reasoning as previously cited above. 

Conclusion 

5. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure, careful consideration should be given prior to applicant's 
response to this Office Action. 
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U.S. Patent 5,988,862 issued Kacyra et al teaches component modeling from picture 
data. 

U.S. Patent 5,894,310 issued to Arsenault et al teaches virtual modeling of systems. 
U.S. Patent 5,812,394 issued to Lewis et al teaches virtual components and installation. 
U.S. Patent 4,937,768 issued to Carver et al teaches virtual system modeling. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Fred Ferris whose telephone number is 703-305-9670 
and whose normal working hours are 8:30am to 5:00pm Monday to Friday. 
Any inquiry of a general nature relating to the status of this application should be 
directed to the group receptionist whose telephone number is 703-305-3900. 
The Official Fax Numbers are: 

After-final (703) 746-7238 

Official (703) 746-7239 

Non-Official/Draft (703) 746-7240 

'Pned'Pevtu. Patent Examiner 
Simulation and Emulation, Art Unit 2123 
U.S. Patent and Trademark Office 
Crystal Park 2, Room 2A22 
Crystal City, Virginia 22202 
Phone: (703) 305 - 9670 
FAX: (703) 305 - 7240 
Fred.Ferris@uspto.gov 

June 2, 2003 




